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This study describes the construction of a questionnaire instrument to measure mathematics 
teacher educators’ knowledge for technology integrated mathematics teaching. The study 
was founded on a reconceptualisation of the generic Technological Pedagogical Content 
Knowledge framework in the specific context of mathematics teaching. Steps in the 
development of the questionnaire were; consideration of the context in which the 
questionnaire would be used, comparison of proposed items with and existing instrument, 
expert review, and pilot testing. The process described provides a model for other 
researchers interested in adapting generic tools for mathematics specific use. 


The Technological Pedagogical Content Knowledge (TPCK) framework (Mishra & 
Koehler, 2006), renamed TPACK (Thompson & Mishra, 2007), was articulated to describe 
teachers’ knowledge of technology integrated teaching. The framework has been 
introduced to explain the knowledge base needed for teachers to teach effectively with 
technology. It stems from the notion that technology integration benefits from a careful 
alignment of content, pedagogy and the potential of the technology (Niess, 2005). Teachers 
who want to integrate technology in their teaching practice, therefore, need to be 
competent in technology, pedagogy and content. Mishra and Koehler (2006) went further 
arguing that effective technology integration in teaching also requires knowledge defined 
by the relationships and interplays between knowledge of technology, pedagogy and 
content as indicated in Figure 1 . 
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Figure 1. TPACK framework (source: Koehler & Mishra, 2009). 


As indicated in Figure 1, the interplay between technology, pedagogy and content give 
rise to i) Pedagogical Content Knowledge (PCK) which refers to knowledge of pedagogy 
that is appropriate to the teaching of specific content; ii) Technological Content 
Knowledge (TCK) which refers to an understanding of the teacher that technology and 
content influence each other for effective learning; iii) Technological Pedagogical 
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Knowledge (TPK) which is the understanding of how teaching changes when particular 
technologies are used, and finally where all three intersect, iv) Technological Pedagogical 
Content Knowledge (TPACK) referring to the knowledge needed by teachers to 
understand the pedagogical approaches that use technologies in systematic ways to teach 
content. 


Measuring TPACK 

Instruments to measure teachers’ knowledge of technology integrated teaching have 
been developed using the TPACK framework in diverse fields of study and a variety of 
contexts (e.g., Abbitt, 2011; Albion, Jamieson-Proctor, & Finger, 2010; Graham, Cox, & 
Velasquez, 2009; Koehler & Mishra, 2005; Schmidt et ah, 2009). One example of a 
questionnaire instrument to measure teachers’ TPACK was described by Koehler and 
Mishra (2005). In that USA study, 4 f acuity members and 13 student participants 
responded to questions about individual and group perceptions of elements of the TPACK. 
The questionnaire instrument comprised 33 Likert scale items and 2 open-ended questions 
and was primarily designed to determine the level of participants’ TPACK. 

A literature review study (encompassing 34 j ournal articles, 52 p apers published in 
conference proceedings, two book chapters, two doctoral dissertations, and one report) on 
instruments developed to measure TPACK in the context of pre-service teacher preparation 
programs by Abbitt (2011), showed the importance of using quantitative and qualitative 
approaches to understanding TPACK. Quantitative methods (e.g., closed response 
questionnaire items) can provide efficient measures whereas qualitative approaches (e.g., 
open response questionnaire items) can provide further insight into respondents’ thinking. 
Examples of instruments include that of Albion et al. (2010) who described the 
development and empirical validation of an instrument to measure and audit the TPACK 
capabilities of final year teacher education students at two universities in Queensland, 
Australia. They found that the teaching with technology audit questionnaire was a valid, 
reliable and multi-dimensional instrument. Graham et al. (2009) used a self-report 
questionnaire and performance assessment strategies for measuring TPACK confidence of 
in-service teachers. The instrument addressed technology related elements of the TPACK 
framework using a total of 3 1 items specific to science. 

The examples described above illustrate the effort that has been made to develop valid 
and reliable instruments which can measure teachers’ TPACK. All were developed based 
on the generic definitions of the TPACK framework (Mishra & Koehler, 2006). The 
authors of the present study, however, argue the importance of understanding and 
explaining the use of the TPACK framework specific to mathematics teaching. They were 
also interested in an instrument that could be used with mathematics teacher educators 
(MTEs) - a group whose knowledge has been under-researched (Zazkis & Zazkis, 2011). 
This paper reports the development of an instrument to measure MTEs’ TPACK. The 
development process, detailed in the following sections entailed conceptualising the 
TPACK framework for mathematics teaching, consideration of context, comparison with 
an existing instrument, expert review, and pilot testing. 

Instrument Development 

Various studies have illustrated procedures for developing reliable and valid 
instruments to measure a given construct (e.g., Colton & Covert, 2007; Liang et al., 2008). 
Colton and Covert (2007) recommended an iterative process involving constant revision. 
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They recommended eight cyclic steps: identifying the purpose and focus of the study; 
obtaining feedback from stakeholders to clarify the purpose and focus; identifying the 
research methodology and type of instrument to use for data collection; beginning to 
formulate items or questions; pre-testing items with content experts and potential 
respondents; making revisions based on f eedback; piloting and revision; and finally 
administering the instrument. They indicated that instrument development ought to be a 
systematic yet creative activity requiring continual refinement and revision. Liang et al. 
described five phases to develop an instrument to evaluate pre-service teachers (PSTs) 
views on the nature of science. These were, i) selection of standards and literature based 
items, ii) pilot tests and interviews, iii) expert review, iv) further revision and v) 
comparison with existing instruments. This study drew on both Colton and Covert’s 
iterative process for instrument development, and the instrument development phases of 
Liang et al. (2008). The process used is indicated in Figure 2. 



Figure 2. Process of developing an instrument to measure mathematics teacher educators’ TPACK. 

This study included context analysis, literature review (to conceptualise TPACK in the 
context of mathematics teaching), expert review, and a pilot test to develop the instrument 
to measure MTEs’ TPACK. 

Development of an Instrument to Measure Mathematics Teacher Educators’ 

TPACK 

This section explains each stage of the development process as shown in Figure 2. 

Conceptualising TPACK in the Context of Mathematics Teaching 

The need to specify TPACK in relation to mathematics teaching in order to provide a 
sharper lens than the generic TPACK framework alone and to understand better the 
knowledge required for teachers of mathematics to use technology in their teaching was 
the starting point. Researchers in mathematics education have conceptualised the 
knowledge needed to teach mathematics in various ways (e.g., Ball, Thames, & Phelps, 
2008; Chick, Baker, Pham, & Cheng, 2006; Rowland, Huckstep, & Thwaites, 2005). 
However, Ball et al.’s (2008) framework with its emphasis on mathematical knowledge 
suited our purpose of developing a mathematics specific framework for technology 
integrated teaching. Hence Ball et al.’s (2008) Mathematical Knowledge for Teaching 
(MKT) and the TPACK framework (Mishra & Koehler, 2006) were used as bases for 
understanding technology integrated mathematics teaching. The TPACK framework was 
restructured by refocussing three new components for teaching mathematics using 
technology (Technology Knowledge (TK), Specialised Pedagogy Knowledge (SPK) and 
Specialised Mathematics Knowledge (SMK) as in indicated in Figure 3. The definitions of 
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each of the knowledge types arising from interplay of the various components are provided 
in Table 1. 


Technological Knowledge (TK) 

A 


Specialised Technological 
Pedagogical Knowledge (STPK) 

Specialised Technological 

Specialised Pedagogical < SPK SMK Mathematics knowledge (STM K) 

Knowledge (SPK) 

Specialised Mathematics 
Knowledge (SMK) 

Specialised Pedagogical 
Mathematics Knowledge (SPMK) 

Context 

Figure 3. Knowledge required in teaching mathematics with technology. 


Specialised Technological and 
Mathematics Pedagogical 
Knowledge (STAM P) 


Table 1 

Knowledge required in teaching mathematics using technology 


Component 

Definition 

SMK 

Actual mathematics concepts and specialised mathematics knowledge 
required for teaching. 

SPK 

Knowledge of various instructional strategies and the knowledge required to 
understand the nature of the target students to teach mathematics. 

TK 

Various technologies, ranging from low-tech technologies to digital 
technologies. 

STMK 

Teaching effective mathematics cementing the special content knowledge of 
mathematics with the application of technology. 

STPK 

Anticipating what students are likely to think and find confusing and 
addressing this with the application of technology and selected pedagogy. 

SPMK 

Pedagogical knowledge that fits with the mathematical concepts. 

STAMPK 

The application of technology for effective mathematics learning of 
students. 


Consideration of context 

The context in which an item is presented has an influence on t he way respondents 
interpret and answer it (Colton & Covert, 2007). The instrument was designed to measure 
MTEs’ knowledge of technology integrated mathematics teaching in the context of initial 
teacher education in Ethiopia. This is in contrast to more common uses of the TPACK 
framework with pre-service and in-service teachers. The items needed to be in relation to 
MTEs (Colton & Covert, 2007). For example, there were items which referred to how 
teacher educators can be models for their PSTs in the use of technology in their teaching 
(e.g., “I can provide leadership in helping PSTs and others to teach ICT integrated 
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mathematics with teaching approaches”). Similarly, the wording of each item was li nk ed to 
MTEs teaching primary PSTs of mathematics. The wording of items was considered 
carefully to ensure that they would be understood by Ethiopian MTEs. 

Comparison with an Existing Instrument 

A number of studies (e.g., Agyei & Voogt, 2012; Chai, Koh, & Tsai, 2010; Koh, Chai, 
& Tsai, 2010) have described the construction of instruments based on Schmidt et al.’s 
(2009) questionnaire to measure teachers’ TPACK. For example, Agyei and Voogt (2012) 
adapted items from Schmidt et al. (2009) to address the integration of spread sheets in 
mathematics teaching, and Chai et al. (2010) adapted the same questionnaire to examine 
PSTs’ TPACK perceptions before and after their ICT course. Koh et al. (2010) used an 
adaptation of Schmidt et al.’s (2009) questionnaire, based on review of the items by 
experts who are specialised in ICT, to examine Singaporean PSTs’ TPACK. There were 
thus precedents for basing a new instrument on that of Schmidt et al. (2009) 

Schmidt et al.’s (2009) instrument was used as a frame by redefining the TPACK 
framework from the perspective of teaching mathematics with technology and modifying 
the underpinning theories in understanding TPACK in relation to mathematics teaching. 
For example, in the definition of CK Mishra and Koehler’s (2006) TPACK framework was 
reconceptualised to account for the distinctive nature of CK needed to teach mathematics, 
as described by Ball et al. (2008) and defined as “Specialised Mathematical Knowledge” 
(SMK). This specialised knowledge assisted in the designing of items related to Content 
Knowledge (CK) later (SMK) of mathematics teachers in the TPACK framework. Table 2 
shows examples of how items related to TK were modified (e.g., one item became four) 
while Table 3 provides examples of modified CK items (e.g., three items became two). 


Table 2 

Modification of items for the Technological Knowledge (TK) construct of TPACK 


Schmidt et al. 
(2009) item 

New items included 

Justification 

I know about 
technologies 
that I can use 
for 

understanding 
and doing 
mathematics. 

1 . I can use a wide range of 
technologies to teach maths. 

2. I can select technologies to use in 
my classroom that enhance what I 
teach. 

3. I can’t think of teaching maths 
without the use of technology. 

4. I know how to cement the 
knowledge needed to teach maths 
with the application of technologies. 

The item in Schmidt et al. (2009) 
was only one which makes it 
difficult to measure teacher’s TCK 
in an item; hence there was a need 
to include more items. In addition, 
the special type of knowledge 
needed for mathematics teachers 
should be measured while 
influenced by the application of 
technology. 


As a result, the definition for each of the newly conceptualised technology integrated 
mathematics teaching components was compared with the items included in Schmidt et al. 
(2009). Based on the comparison, some items were revised, included or even discarded 
based on the new conceptualisation of TPACK, the context, and participants in the study. 
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Table 3 

Modification of items for the Content Knowledge (CK) construct ofTPACK 


Schmidt et al. (2009) item 

Modified item 

Justification 

1 . 1 have sufficient 
knowledge about maths. 

2. 1 can use a mathematical 
way of thinking. 

3.1 have various ways and 
strategies of developing 
my understanding of 
maths. 

1 . 1 have a mathematical 
knowledge unique to 
teaching. 

2.1 understand the difference 
between the knowledge 
required for teaching 
mathematics and common 
knowledge for mathematics. 

According to Ball et al., (2008) 
teaching mathematics demands 
a special kind of knowledge 
different from knowing 
mathematics called Specialised 
content knowledge; hence this 
has to be addressed in the 
questionnaire. 


Creating a Draft Instrument 

The definitions in Table 1, context considerations, and comparison with an existing 
instrument were used to construct the draft instrument for measuring MTE’s knowledge for 
technology integrated mathematics teaching. Additional items were constructed to measure 
each of the constructs that appear in the modified framework (Figure 3) when SMK is 
influenced by the application of technology and selection of a particular pedagogy (namely 
SPMK, STMK, and STAMPK). For example, there was a need for items that can measure 
MTEs’ content knowledge unique to teaching mathematics (Ball et ah, 2008). Examples 
are provided in Table 2. An effort was made to show how this knowledge might interact 
with the application of technology (i.e. STMK component). As consequence, the STMK 
component was stated as “I know how to cement the knowledge needed to teach 
mathematics with the application of technologies”. In a similar fashion a total of 41 items 
were constructed requiring responses on 5 -point Likert scales from Strongly Disagree to 
Strongly Agree. Open-ended questions were added to invite participants to explain their 
experience in relation to each of the items. The qualitative (open-ended questions) and the 
quantitative items (Likert type) helped to ensure the validity of the instrument by ensuring 
that it e licited evidence relating to the constructs intended to be measured (Colton & 
Covert, 2007). 

The draft questionnaire also asked respondents to provide infonnation about their 
experience of teaching at teachers’ college and school level, their access to various 
technologies, and the frequency with which they used these, and their confidence to 
undertake a range of tasks involving the use of technology. 

Expert Review 

The review aimed to pinpoint ways to define and operationalise the constructs and 
indicate whether each item seemed likely to measure what it was intended to (Liang et ah, 
2008). Two experts in mathematics education reviewed the questionnaire and several 
changes were made. Table 4 shows examples of the kinds of improvements made such as 
adaptations to explicitly address MTEs’ work, and splitting ‘double-barrelled’ items. 

In addition, there were recommendations by the experts to add two open-ended items 
asking participants for such things as an example of how they have successfully used 
technology in their teaching of PSTs. As a result, more open-ended items (making a total 
of 9 such items) were included in the questionnaire which could offer detailed and rich 
information about teachers’ knowledge on TPACK. 
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Table 4 

Examples of improved items based on expert re view 


Original item 
I know about technologies 
that I can use for 
understanding and doing 
mathematics (TCK). 

I can anticipate 
mathematical concepts 
what students will find 
confusing and solve with 
appropriate teaching 
method (PCK). 


Expert reviewed items 
I know about technologies that 
I can use to PSTs’ 
understanding of mathematics. 

I can anticipate mathematical 
concepts that PSTs will find 
confusing. 

I can prevent PSTs’ learning 
difficulties with appropriate 
teaching method. 


Justification 
The audience was not 
addressed. There is a need 
to address the audience. 

The item was complex; it 
contained more than one 
item. There was a need to 
split and develop two more 
items for understanding and 
misconceptions 


Pilot Test 

After the instrument was reviewed by content experts, a pilot test with a small group of 
Australian MTEs (5 responses were obtained) was conducted. The pilot-test assisted in 
addressing problems that might occur during administration. Furthermore, this step was 
essential in checking that the items were such that the instrument was to fulfil its purpose, 
and that it was unlikely that participants in the study would misunderstand (Colton & 
Covert, 2007). The pilot testing revealed that some items were vague and too broad. As a 
result, several items were modified. The pilot testing was, also helpful in identifying 
unambiguous and understandable items (Colton & Covert, 2007). For example one open- 
ended question read “In your opinion, what are the challenges to use technology into your 
teaching of pre-service teachers?” In relation to this item, one respondent commented “so 
many". This indicated that the question was too broad and hence, there was a n eed to 
modify the question to focus on a particular issue. 

The final instrument comprised, in addition to items seeking demographic data, closed 
response items asking about the accessibility and frequency of use of several specific 
technologies, types of social media, and software packages, all chosen for their likely 
relevance in the context of the study and each accompanied by a prompt for respondents to 
list others, 41 Likert type items measuring aspects of TPACK followed by space to expand 
upon or clarify any of these responses, and nine open response questions. 

Conclusion 

This paper has described the procedure used to develop an instrument to measure 
MTEs’ knowledge for technology integrated mathematics teaching. It is acknowledged that 
the fact that the pilot test sample differed from the target population is one limitation of the 
study. The study emphasised the importance of conceptualising the TPACK framework in 
the context of mathematics teaching to develop a valid and reliable instrument. It provides 
an approach that might be useful for researchers in other curriculum areas interested in to 
increase emphasis on developing an instrument based on subject specific definitions of 
TPACK to measure its constructs. 


360 


Acknowledgement 

The authors acknowledge the contribution of Associate Professor Rosemary 
Callingham to the research reported in this paper. 

References 

Abbitt, J. T. (2011). Measuring technological pedagogical content knowledge in preservice teacher 
education: a review of current methods and instruments. Journal of Research on Technology in 
Education, 43(4), 281. 

Agyei, D. D. & Voogt, J. (2012). Developing technological pedagogical content knowledge in pre-service 
mathematics teachers through collaborative design. Australasian Journal of Educational Technology, 
25(4), 547-564. 

Albion, P. R., Jamieson-Proctor, R., & Finger, G. (2010). Auditing the TPACK confidence of Australian pre- 
service teachers: the TPACK confidence questionnaire (TCS). Paper presented at the 21st International 
Conference of the Society for Information Technology and Teacher Education (SITE 2010). 

Ball, D. L., Thames, M. H., & Phelps, G. (2008). Content Knowledge for Teaching What Makes It Special? 
Journal of Teacher Education, 59(5), 389-407. doi: 10.1177/0022487108324554 

Chai, C. S., Koh, J. H. L., & Tsai, C. C. (2010). Facilitating preservice teachers' development of 
technological, pedagogical, and content knowledge (TPACK). Educational Technology & Society’, 13(4), 
63-73. 

Chick, H., Baker, M., Pham, T., & Cheng, H. (2006). Aspects of teachers’ pedagogical content knowledge 
for decimals. Paper presented at the In Proceedings of the 30th annual conference of the International 
Group for the Psychology’ of Mathematics Education, Prague. 

Colton, D., & Covert, R. W. (2007). Designing and constructing instruments for social research and 
evaluation (1 st ed.). United States of America: Jossey-Bass. 

Graham, C., Cox, S., & Velasquez, A. (2009). Teaching and measuring TPACK development in two 
preservice teacher preparation programs. Paper presented at the Society for Information Technology & 
Teacher Education International Conference, Charleston, SC, USA. http://www.editlib.Org/p/31297 

Koehler, M. J., & Mishra, P. (2005). What happens when teachers design educational technology? The 
development of technological pedagogical content knowledge. Journal of Educational Computing 
Research, 32(2), 131-152. 

Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary 
Issues in Technology and Teacher Education ( CITE Journal), 9(1), 60-70. 

Koh, J. H. L., Chai, C. S., & Tsai, C. C. (2010). Examining the technological pedagogical content knowledge 
of Singapore pre-service teachers with a 1 arge-scale questionnaire. Journal of Computer Assisted 
Learning, 26(6), 563-573. doi: 1 0. 1 1 1 1/j . 1 365-2729.20 10.00372.x 

Liang, L. L., Chen, S., Chen, X., Kaya, O. N., Adams, A. D., Macklin, M., & Ebenezer, J. (2008). Assessing 
preservice elementary teachers' views on the nature of scientific knowledge: A dual-response instrument. 
Asia-Pacific Forum on Science Learning and Teaching 9(1), Article 1, retrieved from 
http://www.ied.edu.hk/apfslt/download/v9_issuel_files/liang.pdf. 

Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher 
knowledge. The Teachers College Record, 705(6), 1017-1054. 

Niess, M. (2005). Preparing teachers to teach science and mathematics with technology: Developing a 
technology pedagogical content knowledge. Teaching and Teacher Education, 21(5), 509-523. 

Rowland, T., Fluckstep, P., & Thwaites, A. (2005). Elementary teachers’ mathematics subject knowledge: 
The knowledge quartet and the case of Naomi. Journal of Mathematics Teacher Education, 5, 255-281. 

Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technological 
pedagogical content knowledge (TPACK): The development and validation of an assessment instrument 
for preservice teachers. Journal of Research on Computing in Education, 42(2), 123-149. 

Thompson, A., & Mishra, P. (2007). Breaking news: TPCK becomes TPACK! Journal of Computing in 
Teacher Education, 24(2), 38 - 39. 

Zazkis, R., & Zazkis, D. (2011). The significance of mathematical knowledge in teaching elementary 
methods courses: Perspectives of mathematics teacher educators. Educational Studies in Mathematics, 
76, 247-263. 


361 


